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Artificial Intel . genre research l.as come under fire for failing to fulfill its promises. A 
growing number of AI researchers are reexamining the bases of AI research and are 

lation Of " R TrrT im \ thnt *>*h,vior «» be fully explained as manipu- 

* , o n y, 'r b ° S }*>■ algorithms. Three recent books - Mind over Machine (H. Dreyfus 
and S. Dreyfus), Under, landing Com,, liter, and Cognition (T. Wmograd and F. Flores) and 
l rain Savior and liolot. (J AH, us) - explore alternatives and open the d^r toliew 
architectures that may he able to learn skills. 
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Will Machines Ever Think? 

Peter J. Denning 

Research Institute for Advanced Computer Science 
May 16, 1986 

Can machines think? This question has fascinated inventors, philosophers, 
and logicians for centuries. In 1938, the British mathematician Alan Turing 
showed that a simple computational model, now called the Turing machine, was 
capable of universal computation: it could simulate the computation of any 
other Turing machine. This simulation is the basis of stored-program comput- 
ers. It might seem then that stored-program computers would e/entually be 
capable of simulating arbitrary human actions. For with information .bout the 
state of every neuron and the mechanics of neuron firing, it would seem possible 
to compute the next state of someone s every neuron and thus foretell that 
person’s next action. Granted, in practice it may be impossible to acquire suffi- 
cient information for a precise prediction, but in principle it seems possible to 
program machines to act like people. 

In 1950, Turing proposed a method of determining how close a machine 
might be to acting human. His method, now called tne Turing test (ij, 
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envisages a human interrogator communicating with two entities by teletype: 
one a human, the other a computer. How long would it take the interrogator to 
determine which is which? How reliable might the determination be? Turing 
asserted that in Fifty years' time it would be possible to make computers play the 
imitation game so well that 70% of interrogators would not make the right iden- 
tification after Five minutes of questioning. This line of reasoning underlies con- 
temporary approaches to evaluating t lie skill of expert systems. The longer it 
takes a human expert to distinguish the actions of a program from the actions of 
a fellow expert, the more skillful the program is judged. This line of reasoning 
also leads some artiFicial intelligence (A I) programmers to be tolerant of pro- 
grams t hat make mistakes, as long as the error rate is no worse than that of 
human experts. 

Turing did well to replace the question, Can machines think? with the ques- 
tion, How well can a digital computer program imitate people? The terminology 
of the original question is too open to endless philosophical inquiry. What is a 
machine? What is thought? Turing offered a related, but not equivalent, ques- 
tion that could be resolved scientifically by experiment. An experiment to 
answer the original question is about as meaningful as an experiment to deter- 
mine whether I have a soul. 

How good is the substitute question? Will computer programs ever mimic 
human actions faithfully? It is important to realize that, while logic and science 
may help shed light on this question, it cannot be settled purely by argument or 
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by experiment. 

AI research has had many years’ experience tussling with this seemingly 
more answerable question. In 1907, Marvin Minsky of MIT speculated that a 
computer with a properly organized set of one million facts about everyday life 
should be able to exhibit very great intelligence. Today’s computers can easily 
store several times this amount of information and yet no one has succeeded in 
constructing an intelligent program. Around 1970, chess master David Levy 
challenged predictions that chess programs would soon play at master levels. He 
offered a prize to anyone who could write a computer program within ten years 
that would beat him; to this day, no one has constructed a program that can 
win more than one or two games out of ten against Levy. Around 1980, the 
Japanese began asserting that they would be able to construct many computer 
programs possessing expert levels of skill in selected domains. Will they succeed? 

intelligence has been a moving target for simulators. Most people distin- 
guish between a complex mechanism and the thought process that created it. 
(Patent law makes this distinction as well.) Automatic pilots are expert fliers, 
but they are not pilots. Computer programs that search four moves ahead may 
be formidable chess players, but they are not masters. In 19GC, Joseph Weizen- 
btium of MIT wrote a program called Eliza, a version of which simulated conver- 
sations of a psychologist (2). Eliza performed simple transformations on prede- 
fined keywords in the input, text; in response to “I’m feeling A',” it might type, 
“Why are you feeling XI" With Eliza, Weizcnbauin demonstrated that it is pos- 
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si bio to make convincing demonstrations of intelligence without giving the pro- 
gram any real understanding at all. As soon as someone finds a way of mechan- 
izing a class of human actions, people tend to stop regarding those actions as 
intelligent; indeed, they often want the machines to take over the tedious rou- 
tine! 

It is disquieting to a growing number of AI researchers that the fundamen- 
tal question of their field is so slippery. The elusiveness of answers has brought 
out the skeptics. David Parnas and John Shore have criticized the trial-and- 
error approach to programming expert systems; Hubert and Stuart Dreyfus have 
asserted that after 25 years AI has failed to live up to its promises, and there is 
no evidence that it ever will; Douglas Hofstadter says that the way many AI 
questions arc posed is shallow. A growing number of A I researchers ask, Why is 
it so difficult to elicit intelligent behavior from machines? Is it poor engineering? 
Overly ambitious goals? Or is there something fundamental that puts the goals 
beyond our reach? The continuing debate may produce a new perspective on the 
role of machines in human society. 

Three recent books shed intriguing new light on these old questions. The 
first, Mind over Machine , by Hubert and Stuart Dreyfus, analyzes the ability of 
machines to achieve human skills (5). The second, Under standing Computers and 
Cognition, by Terry Winograd and Fernando Flores, analyzes the possible rela- 
tions between humans and machines in search of new foundations for computer 
design (4). The third, Drains , Behavior , and Robots , by James Albus, analyzes 
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the characteristics of intelligent behavior and computing architectures required 
to simulate them (5). As you will see, the books shed light on each other as well, 

The Dreyfuses set out to show that human beings have intelligence that 
machines cannot match. To support their argument, they analyze human skills, 
an important aspect of intelligence, distinguishing five levels: novice, advanced 
beginner, competent performer, proficient performer, and expert. The novice 
knows basic facts about a subject and context-independent rules for using those 
facts. The advanced beginner is able to recognize a few common situations and 
perform associated actions, and is able to take context into account. The com- 
petent performer experiences a high level of personal involvement in the subject; 
his behavior is strongly goal-oriented. He is able to act without conscious 
thought about the rules and to reason analytically about situations. The profi- 
cient performer is able to recall whole situations and apply them to the present 
without having to decompose them into components. The expert is fully 
involved in the situation, ha little awareness of the skill, and makes little cons- 
cious use of analytic reasoning; when exercising the skill, he operates in a larger 
world where he visualizes and manipulates whole objects and situations. 

A good example that occurs to me to illustrate these points is the skill of 
writing. The novice depends on books detailing the rules of grammar, punctua- 
tion, and spelling; he finds writing correct sentences and paragraphs a tedious 
process requiring frequent reference to the rule books. The advanced beginner 
knows most of the major rules and can write sentences and paragraphs without 
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having to look much up. The competent writer thinks more about the act of 
communicating than about the details of writing; he has learned rules of style 
and applies them automatically, and he is good at constructing and using out- 
lines. The proficient writer feels completely at ease with writing; he seldom 
makes mistakes in grammar or style; he is able to analyze and edit his own writ- 
ing; he is able to imitate the writing styles of others; he makes good use of fig- 
ures of speech and other rhetorical, devices; he is able to plan and execute large 
works. The expert writer lives in a world of ideas that he feels an urge to com- 
municate. He is constantly formulating ways to say new things and new ways to 
say old things; he is able to communicate moods; he can imitate many styles of 
writing but he has his own style so distinctive tli at he can detect instantly when 
an editor has changed a single word of his work. The expert can formulate large 
works in his head, moving easily between general and detailed views. 

In this hierarchy, manipulation of rules and symbols is characteristic of the 
lowest skill levels; the recall of highly abstract patterns is characteristic of the 
highest skill levels. The rules for the lowest levels can be written down unambi- 
guously and used immediately by a novice; the skills of the highest levels are 
learned only through many repetitions. The Dreyfuses refer to the ability to per- 
form at high levels as know-how, which they distinguish from the ability to 
manipulate symbols and rules. They argue that the stored-program computer, 
which is good at manipulations of rules and symbols but incapable of fast com- 
putations of whole patterns, is unlikely to pass much beyond the stage of bare 
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competence. According to their account, entirely different forms of computation 
are involved in know-how, and the only existing device with some of the required 
properties of know-how’ is the hologram, where the entire image is encoded into 
every local region and where similarities can be determined by passing light 
through two holograms. Virtually nothing is known about the architecture or 
function of pattern computers, least of all whether they would have the human 
power to form images that are a! factions of other images. 

The Dreyfuses conclude that stored-program computers cannot perform at 
high levels of skill because their internal structure is incapable of processing 
information in the same way human experts do. They warn that there is a' lurk- 
ing danger in research programs founded on the belief that today's computers 
can be smart: rather than waste the large investment, people will try to use 
expert systems anyway. Merely competent programs will be deployed where 
nothing less than full expert skill is required. They also say that expert systems 
used to teach or train may limit the progress of students to rule and symbol 
manipulation, whereas true expertise moves well beyond that, into know-how. 

Like the Dreyfuses, Winograd and Flores are troubled by the inability of 
modern AI research to fulfill its promises. By analyzing the framework in which 
research is pursued, the nature of the questions being asked, and the approaches 
to these questions, they hope to open a new’ perspective. 

According to their view, the basic questions of interest in computing are 
questions of design: What pu"^oses do computers serve in the context of human 
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practice? Design involves not just how computers operate but how they affect 
people. For example, a word processor looks different to different observers: to 
its manufacturer, it looks like a box of electronics; to its programmers, it looks 
like a package of software; to writers, it looks like a tool for effective communica- 
tion. To all three, it is most visible when it breaks down. 

Winograd and Flores find that our approach to design is strongly influenced 
by our tradition: the entire context of experience and ways of viewing the world, 
the common set of unspoken assumptions. Tradition is concealed by its gratui- 
tousness. The Western scientific tradition — scientific method — holds that 
knowledge is objective; it values explicit theories that can be systematically used 
for predictions about parts of objective reality. Winograd and Flores call this 
the rationalistic tradition. The computer tradition, which derives from the 
rationalistic tradition, values information, representation, and decision-making; 
it also emphasizes analogies with the brain and intelligence, as well as other 
anthropomorphisms. 

The rationalistic probiem-solving process, held in high esteem by Western 
scientists, consists of three parts: give a precise statement of the situation in 
terms of objects and relationships among them; enumerate alternatives for solu- 
tion; and evaluate each alternative until one of sufficiently low cost is found. 
Stored-program computers are ideally suited to this process: objects, relation- 
ships, and rules can be represented unambiguously as programming-language 
constructs; methods such as deduction and search can be used to enumerate 





alternatives; calculations and comparisons can be used to select a low-cost alter- 
native. Much of current research in AI has a fundamental orientation that 
equates intelligence with rationalistic problem solving using heuristic procedures. 
If these are the only elements of human cognition, Winograd and Flores ask, 
then why are mindlike properties attributed to computers but not to other dev- 
ices that provide or process information? 

Considerable evidence is presented by Winograd and Flores to show that 
cognition is not based only on systematic manipulation of representations. The 
Dreyfuses’ argument about skill levels and know-how illustrates the point. 
Rationalistic problem solving is at best a partial description of how people solve 
problems: people get thrown into situations and react. The world is constantly 
being interpreted by the organisms in it. The structure of a given organism 
plays a central role in determining the interpretations of which that organism is 
capable, and similar organisms evolve interpretations that have enough in com- 
mon to allow communication among them. Here is where the crucial difference 
between a stored-program computer and a person show's up: a computer 
processes symbols without regard to their meaning; a person processes every- 
thing within a framewmrk of interpretations. Stored-program computers have 
the wrong structure to compute like people. 

Winograd and Flores call for a new orientation: let’s ask not how to make 
computers behave more like people but how to design computers to help people 
do things more effectively. Some of the things computers are especially good at 
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are executing algorithm;:, retrieving information, processing and filtering signals, 
assisting with communication and monitoring processes. Machines used in these 
ways can have a significant positive effect on society. Design processes that 
emphasize creative ways to use machines to help people are likely to succeed. 

Winograd and Flores’ book leaves an impression that searching for 
machines that think or understand is futile. The Dreyfuses are not so sure: 
although they feel that stored-program computers cannot achieve mindlike pro- 
perties, they leave open the possibility that other architectures might. 

Aibus is much more confident that mindlike behavior can be elicited from 
machines. He is mostly interested in machines whose architecture resembles the 
human nervous system. (This is reminiscent of the argument of Winograd and 
Flores: structure determines function.) In 1950. Turing discussed two approaches 
to building machines that compete with people in intellectual fields. The first is 
♦o pursue very abstract activities, such as logical deduction or chess. The second 
is to provide the machine with sophisticated sensory organs and a mechanism for 
storing complex codes denoting moments of experience: by repeated exposure to 
sensory patterns, these machines could come to recognize familiar patterns and 
perform associated actions. Aibus says that most AJ research has followed the 
former path, perhaps because many of the early workers were mathematician.* 
skilled in abstract reasoning. Virtually the entire effort in AI has been dedicated 
to modeling the abstract reasoning power of the thinking mind. 
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Albas suggests a way of following Turing’s second approach. Rati, -r than 
start at the top of the hierarchy of biological functions, he would start at the 
bottom, reproducing the control functions and beha iOra.J patterns of insects, 
birds, mammals, and primates. He would use these as building blocks for th** 
higher-level functions of the human being, ultimately coming to understand the 
mechanisms that give rise to intelligence and abstract thought in the human 
brain. As a step in t is direction, he analyses the components of the human ner- 
vous system and describes a Cerebellar Model Arithmetic Computer, on which he 
holds a patent, as an architecture capable of reproducing man v of the nervous 
system s behaviors. Much of his book is devoted to showing how this computer 
could be taught and. after learning, how it could carry out functions of various 
nervous systems and react to complicated stimuli. He e\ 2 n shows how it could 
be taught to act as a stored-program computer. 

I find the arguments ot these writers refreshing and exciting. Rather than 
close the door on A I research, they reveal new doors. Some of their explorations 
will lead to new computers potentially capable of processing information more 
like human experts. Others will lead to better formulated design questions. But 
I doubt t licit the debate about whether computers can simulate thinking will ever 
end. Perhaps stored-program computers can’t ever think, but maybe another 
kind of computer .... 
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